The present invention relates generally to data processing and more particularly to overlapping data transfers with data processing in columnar data processing and storage systems.
Columnar data systems typically store, retrieve, and process data in columns in order to increase data throughput. For example, in a columnar database, all of the data values for the first column are stored together followed by all of the data values for second column, and so forth. The data is typically stored in record order, such that the 100th entry for the first column and the 100th entry for second column belong to the same input record. Storing the data in columns enables retrieving selected data columns involved in a query (e.g., customer name and address) as a group, rather than accessing each record row-by-row and extracting the selected data columns from those records.